A Controller for Online Uncertain Constraint Handling

An online problem is a problem which grows over time and such that partial solutions are to be generated before the complete problem is known. Moreover, if the problem is an optimization problem, partial solutions must be aimed at optimizing the overall final solution. There may be some uncertain knowledge on how the problems develop. How should we make intermediate decisions? Can we extend existing constraint handling techniques? In Online Uncertain Constraint Handling (OUCH!), we assume that the problem starts with a conditional constraint optimization problem (CCOP). At each time step, an extension to the CCOP may arrive; that is, a set of variables, constraints and utility functions. Each variable will have a decision deadline, and a decision on that variable must be committed to by that deadline. We assume that decisions cannot be revised once they have been committed to, and also that there is no benefit in making an early commitment. The CCOP will allow us to ‘reject’ variables, will state what that means for each constraint, and will determine the appropriate reward. The objective will be to maximize the total reward over some (possibly infinite) time interval. Specifically, at each time step, we must decide what to do with the variables whose decision deadline has arrived, balancing the immediate reward with the potential for future rewards. If we have a probability distribution for the CCOPs that arrive at each timestep, we can express the future reward in terms of maximum expected utility. The best decision for a set of variables at time step i is: